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ABSTRACT 

This report presents the results of the evaluation of 
the 1988-89 Comprehensive Instructional Management 
Systems-Communication Arts (CIMS-CA) program, which focused on 
developing a holistic ccMimimication arts curriculum and staff 
development program reflecting the most recent research for students 
in kindergarten through eighth grade; corresponding assessment 
instruments; cuid a computer management system. The report notes that 
over 50,000 students participated ia the program during 1988-89 and 
that the curriculum and test component integrated four content areas 
in communication arts — reading, writing, listening, and speaking. The 
report also notes that participation in the drama component was aljout 
7,000 students. In particular, the report documents the level and 
quality of program implementation and assesses the program's impact 
on student achievement in reading and writing. The report is in five 
chapters! (l) introduction; (2) Program Components (which discusses 
project components and implementation); (3) Reactions to, and 
Perceptions of, the Instructional Component; (4) student Outcomes; 
and (5) Conclusions and Recommendations. Twenty-two tables of data 
are included. Eight appendixes contain tables with analyses of 
ciMS-CA writing samples. (SR) 
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EVALUATION SUMMARY 



The goals of the New ¥ork City Board of Education's 
CoiBprehensive Instructional Management Systeas-CoBimanication Arts 
(CIMS-CA) project are to develop *i holistic ctaaaunicatlon arts 
curriculua which reflects the laost recent research for 
kindergarten through eighth grade, a staff development program, a 
corresponding test oofiq;K>nentf and a cosputer sanageaent system. 
Begun in 1980, the project has coi^leted its ninth year. The 
• curriculim and test component, develop^ by teachers, integrates 

four content areas in ccHamunication arts— reading, writing, 
listening, and speaking. The drama component, added during the 
1985-86 school year, continues to be an ii^>ortant part of CIMS-CA 
learning activities. The computer management system is being 
expanded to additional districts through the \ise of personal 
computers (PCs), with C.S.D.s 11 and 30 currently participating 
in this component. 

The total operating budget for the 1988-89 school year was 
$1.4 million, which was about the same as the previous year. 
Funding came from the New York City Board of Education and 
participating community school districts. 

Participation in CIMS-CA remained at about last year's 
level. Community School District 11 entered its senond year of 
mandating project participation for all language arts classes in 
kindergarten through grade eight. Since 1985-86, C.S.D. 8 had 
mandated CIMS-CA for all its seventh and eighth grade classes. 
While there is no direct mandate in C.S.D.s 3, 9, 15, 17, and 30, 
the CIMS-CA curriculum is used extensively. 

CIMS-CA program objectives for the 1988-89 school year were 

to: 

• introduce the computer management PC system to C.S.D. 30, 
and expand its use in C.S.D. 11; 

• continue expanding the computer management system by 
refining PC software; 

• continue efforts to involve school administrators and 
parents in project implementation; 

• continue to provide staff development training for teachers 
and administrators in elementary and junior high schools in 
C.S.D.s 3, 8, 9, 11, 15, 17, and 30; 

• plan for growth of the CIMS-CA project in ways which 
accommodate the original goals of the project. 

The New York city Board of Education's Office of Research, 
Evaluation, and Assessment/Instructional Support Evaluation Unit 
(O.E.R.A./I.S.E.U. ) conducted an evaluation of the program using 
both quantitative and qualitative methods. Highlights of the 
major findings were as follows: 
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Teachers continued to find staff development training *'^ry 
helpful. They most appreciated the after-school works»*ops, 
demonstration lessons, individual conferences, and classroom 
observations. Teachers wanted sore time vith staff 
developsent aanagers. This vas especially true in C.S.D. 
11, %^ere most training was conducted by district teacher 
trainers. Although staff devel^nsent managers expressed 
concerns about the efficacy of only having indirect contact 
with teachers, there is not sufficient evidence at this time 
to make a definitive statement about the C.S.D. li training 
model. 

There was increased involvement of administrators in the 
training workshops, &nd they generally supported the project 
in many schools. CIMS-CA staff and teachers agree that even 
greater administrator involvement would further improve 
project implementation. 

The computer management system staff made substantial gains 
toward the project goal of having all CIMS-CA C.S.D.s access 
the assessment component through use of PCs. C.S.D.s 11 and 
30 used PCs with newly developed software in 1938-89, with 
plans for continued use next year. 



In its fourth year, the drama component was view i as 
successful by CIMS-CA staff and teachers. The drama 
component served over 7,000 students, about the same number 
as last year. Most C.S.D.s were able to provide adequate 
transportation to the theater this year. 

Teachers in mandated C.S.D. 11 were as enthusiastic about 
the CIMS-CA program as teachers in other mandated and 
nonmandated C.S.D.s. Analysis of data on satisfaction from 
C.S.D. 11 did not vary significantly from all other C.S.D.s. 
C.S.D. 11 is the only participating C.S.D. which has 
mandated CIMS-CA for all grades, kindergarten through eight. 

m general, all grade four through eight students achieved 
overall statistically significant mean gains in reading when 
1988 reading scale scores were compared to 1989 scale 
scores. These gains represented moderate to large effect 
sizes and were consistently larger for grades four and five 
than for grades six through eight. 

For grade three, there was a consistent drop in N.CE. 
scores for all C.S.D.s. However, the overall mean loss of 
4.8 N.C.E. points only represented a small effect size, it 
should be noted that only N.C.E. reading scores were 
available for third grade comparison because of the need to 
equate scores obtained from the 1989 D.R.P. reading test and 
for the 1988 M.A.T. reading test. Note also that a new 
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equating table was used this year since a new nora was 
established for the D.R.P. test. 

• ^ere was an overall statistically significant increase in 
CIMS-CA students' writing scores of .8 points from the fall 
of 198S to the spring of 1989. This mean increase 
represented a soderate effect size. Grades kindergarten 
through two showed an educationally meaningful sean gain of 
1.4 points. Grades thr«e through six showed a aean gain of 

• .7 points representing a moderate effect size, and grades 

seven to eight showed a mean gain representing a low effect 
size. 

Based on thefis findings and other inforiiation presented in 
this report, the following specific recoBoaendations are made: 

• Continue to emphasize after-school staff development 
workshops, with ongoing efforts to include all teachers who 
wish to learn how to use the project components, and more 
experienced taachers wishing to sharpen their skills. 

• Investigate the efficacy of the C.S.D. 11 training model, 
where CIMS-CA staff train district teacher trainers to 
conduct staff development workshops. 

• Continue to expand usage of the computer management system 
so that all C.S.D.s will have access to the assessment 
component at some future time. 

• Consider reinstating work on the ninth and tenth grade 
curriculum and assessment component. 

• Continue to involve more school administrators so that 
project implementation can be further enhanced. 

• Continue to implement the highly successful drama component. 

• Continue to involve teachers directly in curriculum and 
assessment design, and to base its development on the latest 
research and knowledge. 

• Further investigate the effect of large-scale mandating of 
the project in entire C.S.D.s and its implications for 
training, distributing materials, and monitoring appropriate 
use. 

• Continue vigilance keeping the integrity of the original 
goals and objectives of the project, in face of the large- 
scale expansion of the past several years. 



iii 
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I . INTROmJCTION 



PROGRAM BACKGRQOND 

The Comprehensive Instructional Management Systems - 
Communication Arts (CIMS-CA)* project began its ninth year in 
1988-89. The primary goals are to develop: a holistic 
communication arts curriculum and staff development program that 
reflects the most recent research for students in kindergarten 
through eighth grade, corresponding assessment instr\aments , and a 
computer management system. The instructional curriculum a 
assessment instruments integrate four content areas: reading, 
writing, listening, and speaking. The computer management system 
is designed to help organize and coordinate materiij's and 
assessment outcomes. 

For 1988-89, the CIMS-CA project budget remained about the 
same size as for 1987-88, — approximately $1.4 million. The 
project's budget reflects grants from the New York City Board of 



*This project, begun in 1980, was formerly called the 
Instructional Management Project in Communication Arts. For 
more information about project implementation in previous years, 
see: New York City Public Schools' "An Evaluation on the 
Instructional Management Project in Communication Arts," MEDARP, 
1981; "An Evaluation of the Instructional Management Project- 
Communication Arts," MEDARP, 1982,* "An Evaluation of the 
Instructional Management Project - Communication Arts, 1982-83," 
MEDARP, 1983,' "Comprehensive Instructional Management Project - 
communication Arts, 1983-84," Instructional SuK>ort Evaluation 
Unit, 1984? "Coaiprehensive Instructional Management Project - 
Communication Arts, 1984-85," Instructional Support Evaluation 
Unit, 19851 "Comprehensive Instructional Management System 
(CIMS) -Communication Arts, 1985-86," Instructional Support 
Evaluation Unit, 1987? "Comprehensive Instructional Management 
system (CIMS) -Communication Arts, 1986-87," Instructional 
Support Evaluation Unit, 1988? and "Comprehensive Instructional 
Management System (CIMS) -Communication Arts, 19e7'88, 
Instructional Support Evaluation Unit, 1989. 



J2 



Education, as well as funding from participating community school 
districts. 

During the past eight years of the CIMS-CA project, 
irricttlUB and assessaent measures, as well as a computer 
sunagenent system, were developed for grades kindergarten through 
eight. Additionally, a very successful drama component, serving 
over 7,000 students, was i^lemented for grades four through 
eight. Kork on a ninth and tenth grade curricultun, begun in 
1986-87, has been temporarily halted due to staffing and funding 
priorities. 

Overall project goals included: making the project 
available to as many teachers and students as possible, and 
maintaining the integrity of project goals at a time when usage 
has increased dramatically over the past several years. Curricula 
goals for 1988-89 were to maintain the high level of curricula 
implementation, and continue and expand staff development for 
teachers and administrators. No new curricula materials were 
developed or implemented for this year. Assessment goals for the 
1988-89 school year included maintaining the computer management 
system in C.S.D. 15, expanding the system to C.S.D.s 11 and 3 0, 
and providing workshops for teachers and administrators in using 
the completer management system. 

Since its inception, the project has emphasized the use of 
staff -developed materials for curriculum and assessment. Direct 
involvement of teachers in the development, implementation, and 
assessment processes have made staff development a project 
priority. Use of the CIMS-CA curricula has, from the start, been 
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limited to teachers who have participated in staff-devi ^pment 
workshops. AS participation in the project has increased over 
the years, project staff have continually faced a ijreater demand 
for staff "development craininc. 

Additionally, continuing efforts to further involve 
adi^inistrators and parents once again received priority attention 
in all districts this year. Making substantial progress in some 
districts, project staff have been encouraged to pursue efforts 
to involve all professional staff and parents so that learning 
can be more efficient and effective. 
P<;>P^i|;atiqn SERVED 

For 1988-89, the CIMS-CA project continued to serve 
kindergarten to grade eight students in C.S.D. 11 as mandated by 
the district office in 1987. For the third year, selected junior 
high schools in C.S.D. 8 were mandated to use the project 
curriculum. A high level of participation continued in C.S.D.s 
3, 9, 17, 15, and 30. C.S.D.s with informal links to the program 
included 13, 18, 20, and 24, as well as several nonpublic 
schools . 

In 1988-89, over 50,000 students participated in the 
project, about the same number as last year. Students were 
served '^y approximately 1,100 teachers, also similar to 1987-88. 
Participation in the drama component remained at about 7,000 this 
year. 

PROGRAM OBJECTIVES 

CIMS-CA program objectives for the 1983-89 school year were 

to: 
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• intrtKiuce and expand the computer management system to 
C.S.D.s 11 and 30 with the use of PCs and newly developed 
software; 

• continue decentralissing the computer management system by 
refining PC software; 

• continue efforts to involve school administrators and 
parents in project is^lementation; 

• continue to prov *h staff development training for teachers 
and administrator in elementary and junior high schools in 
C.S.D.s 3, 8, 9, 11, IS, 17, and 30; 

• plan for growth of the CIMS-CA project while maintaining the 
integrity of the project's original goals. 

PROGRAM EVALUATION 

The purpose of the 1988-89 evaluation by the Office of 
Research, Evaluation, and Assessment/Instructional Support 
Evaluation Unit (O.R.E.A./I.S.E.U.) was to document the level and 
quality of program implementation and to assess the program's 
impact on student achievement in reading and writing. 

The evaluation focused on the program ' s activities in 
kindergarten through eighth grade in C.S.D.s 3, 8, 9, 11, 15, 17, 
and 30. It noted the increased expansion of the computer 
management system. Particular attention was paid to the mandated 
use of the curriculum in C.S.D. 11. The following "methods were 
used to evaluate the program: 

• Interviews were conducted with project staff members, 
including the project director, research manager, and staff 
development managers. The interviews focused on describing 
and/or assessing each staff member's activities; project 
goals and administrative staff's perceptions of the 
achievement of these goals; program operation in each 
district! staff development efforts; assistance offered to 
teachers and administrators; and curriculum and test 
development and test use efforts. The interviews also 
focused on the identification of the program's strengths and 
weaknesses and recommendations for the following year. 
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• h teacher users* survey was sent to teachers who had used 
tJie CIMS-cyv curriculua, and a total of 148 teachers 
responded. A separate questionnaire was sent to teachers in 
C.S.D. 11 where the program was mandated for the second year 
and 42 teachers responded. The teacher users* survey 
focused on teachers • experience with the CIMS-CA curriculum; 
their perceptions of its usefulness i plans for using the 
curriculum in the future i ratings of staff develosment 
activities and of the drama component? and identification of 
the progreaQ*s strengths and weakjr^esses. 

• Students* scores on standardized reading tests and 
holistically scored writing samples were analyzed. 

SCOPE OF THE REPORT 

This report presents the results of the evaluation of the 

1988-89 CIMS-CA program in six chapters. Project components and 

implementation are discussed in Chapter II. Staff reactions and 

perceptions of the instructional component are presented in 

Chapter III. In Chapter IV, student outcome data are reviewed, 

and conclusions and recommendations are offered in Chapter V. 



II. PROGRAM €X»!PONENTS 



During the 1988-89 school year, cms-CA operated with eleven 
staff laeiabers, a decrease of three from last year. The project 
director, research manager, and nine staff development managers 
comprised the project staff. The CIMS-CA project director 
supervised the project and disseminated information about its 
activities. The research manager supervised the computer 
management system and arranged for staff development sessions 
focusing on its use. The staff development manager who had 
worked on the ninth and tenth grade curriculum during the 
previous year did not return to tne project, causing a temporary 
delay of development and implementation of the ninth and tenth 

grade curriculum. 

The role of the stnff development manager was to recruit 
newly interested teachers and administrators, encourage parent 
participation, conduct staff-development sessions both during the 
school day and after school, provide demonstration lessons, and 
observe teacher lessons and suggest and help with resource 
development . 

The teacher-developed CIMS-CA curriculum for students in 
kindergarten through grade eight integrates all four content 
areas in communication arts — reading, writing, listening, and 
speaking. The curriculum provides concise educational objectives 
developed around thematic material that can be easily integrated 
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with other subjects. In addition* it includes concrete teaching 
strategies. During the 198 5*86 school year, a drama component 
eviahasizing listening, writing, and speaking was added to the 
fourth through eighth grade curricula. From 1986-87 through 
1988-89, the draaa component was expanded with the help of a 
special grant from the New York city Board of Education, the 
number of student participants increased from 3,000 in the first 
year to its presei\t 7,000 for 1988-89. 

Over the past several years tihe curriculum and assessment 
measures for grades kindergarten throu<gh «ix have been revised by 
teacher developers with the aid of staff development managers, 
the research manager, and professional consultants. Curricula 
and assessment measures for grades seven and eight were added 
from 1985 to 1987, and a high school curriculum comportent was 
piloted in 1^88. 

For 1988-89, the main focus w«is the expansion of the 
computer management system, the key aspect of the assessment 
component. No new curriculum development or revision was planned 
for this year. 

COMPUTER MANAGEMENT SYSTEM 

The computer management system provides teachers with prompt 
test results, facilitates instructional planning, and gives 
administrators with useful information for monitoring 
communication arts instruction for long-range educational 
planning. The CIMS-CA computer management system was begun in 
1984-85, with further development and expansion taking place 
during succeeding years. 

7 
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For each placeiaent instjrument and each end-of-theae test on 
every grade level, class matrices and individual student reports 
in reading, listening, speaking, and writing can be generated. 
In addition, parent reports, archive reports, and end-of-year 
reports can be obtained. These reports provide data on students' 
nastery of outcoaes, teacher ratings on listening/speaking 
perfonaance tasks, and writing sample ratings. 

The system is presently operating using two modes: 1) a 
mainframe operations system using the central Board of Education 
computer, with terminals located in the C.S.D. 15 office; and 2) 
personal computer systems (PCs) located at C.S.D. 11 schools and 
the district office. 

The mainframe system was the original mode, begun in 1984- 
85. A systems analyst, provided by the New York City Board of 
Education, operates the computer management system from the 
C.S.D. 15 office. This operations system is used for analyzing 
C.S.D. 15 data, and for overall CIMS-CA data. Until 1987-88, 
when the PC system (PC-CIMS-CA) was operationalized, end-of -theme 
test data was only available to schools in C.S.D. 15. 

Beginning in the 1986-8? school year, plans were made to de- 
centralize the computer management system. Software specialists 
provided by the Board of Education began writing programs for 
PCs, so that individual schools could enter and analyze their own 
snd-of-theme, placement tests and, other relevant data. By 1987- 
88, a PC system was in operation in C.S.D. 11 for use with 
seventn and eighth grad^. test scores. This year, 1988-89, 
software for the elementary level in C.S.D. 11 was scheduled for 
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use. Project research staff offered workshops en data management 
in C.S.D.6 3, IX, 15, and 30. A sajor objective of the CIMS-CA 
project is to make the computer management system, using PCs (PC- 
CIKS) , available to all participating CIMS-CA districts upon 
request. 

In 1988-89, the PC-CIMS-CA project was largely successful in 
its second year of ovaration for C.S.D. 11 junior high school 
data. The software worked well, and school staff had been 
trained to a sufficient lev»l of competence to use the system. 
There were some problems in generating and disseminating outcomes 
in a timely manner, but improvements were seen toward the end of 
the school year. 

Software for the elementary grades was somewhat problematic, 
resulting in a delay in its use in C.S.D. 11 this year. Efforts 
continued throughout the year to solve software glitches. 
Complete u .a of the data management system for elementary levels 
is planned for 1989-90 in C.S.D. 11, as well as for C.S.D. 30. 
The project has planned for additional implementation of the PC- 
CIMS-CA system, at districts » requests. 
STAFF DEVELOPMENT ACTIVITIES 

During the 1988-89 school year, CIMS-CA staff developiaent 
activities included in-school and after-school workshops, rite 
visits, dejionstration lessons, observations, coordination of 
assessment component, organizing special events including the 
drama component, and encouraging further participation by 
administrators and parents. Particular attention was focused on 
receiving feedback on the sixth grade curriculum, revised in 
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1987-88. Also of primary ir.terest was eactending the use of CIMS- 
Ch and CIMS-CA strategies in English as a Second Language, and 
Special Education classes. Supporting the assessment component 
in C«S«D.s 11 and 15 and helping prepare for its use next year in 
C.S.D.. 30 were also of major concern for CIMS-CA staff 
development personnel. 

Staff Development T raining in C.S.D. 11 

In 1987-88, all language arts teachers in C.S.D. 11 were 
mandated to use the CIMS-CA curriculum, although not exclusively. 
CIMS-CA project staff was responsible for training teachers and 
administrators to use the curriculiim and assessment materials 
effectively. 

The two project staff development managers assigned to 
C.S.D. 11 — one for junior high and one for elementary grades — 
began a training program this year for school-based supervisors. 
These school-based trainers were then expected to provide 
workshops for teachers and administrators, based on the 
information gained from CIMS-CA project staff. This system of 
CIMS-CA staff developers training school-based supervisors to 
hold workshops for teachers was new for 1988-89, and unique to 
C.S.D. 11. 

Reaction to this system by the CIMS-CA staff was mi. ad. 
While the overall assessment of the system was positive, there 
was concern that new CIMS-CA teachers needed more direct training 
by CIMS-CA project staff. In 1987-88, there were six CIMS-CA 
staff developers working in C.S.D. 11. This year there were two, 
supplemented by 32 school-based trainers. Further, most teachers 
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attended workshops during the school day, when interruptions and 
scheduling complications made training xsore difficult. While 
project staff agreed that their sessions with school-based 
trainers went well, they were not able to monitor all the 
%/ork8hops for new CXHS-CA teachers. This represented the first 
time in the history of the project where the quality of training 
was out of the direct control of CIHS-CA project staff. 
Staff DBVB }9Pmn% Act:ivities in C.S.D.s 3. 8. 9. 15. 17, and 30 

Staff development activitiea in these districts continued to 
focus on workshops, demonstration lessons, observations, 
coordination of CIMS-CA related activities, and resource 
provision. Ongoing efforts by staff developers to get more 
administrators and parents involved persisted, with some progress 
reported in all C.S.D.s. An increasing number of assistant 
principals and teacher- trainer supervisors attended after-school 
workshops for new CIMS-CA veachers. Additionally, special in- 
school meetings and workshops were held specifically for 
administrators in several districts. In most schools, parents 
became involved in the drama component, and in many districts, 
parents provided ongoing support for the project. Increased 
involvement by school administrators and parents is still viewed 
as an important objective by staff developers. 

Focused attention on cooperative learning within classrooms 
became an important objective of CIMS-CA staff development 
managers this year. Teachers encouraged children to work in 
small groups while carrying out assignments related to the 
curriculum. Increased efforts to make the curriculum more 
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accessible to special education, English as a Second Language, 
and bilingual classes often vent hand in hand with the emphasis 
on cooperative learning. With these students especially, the 
opportunity to read to each other, retell what they hear, and 
help each other with reading coa^rehension assignments enhanced 
the learning process. 

In C.S.D.s 8 and 9, all seventh and eighth grade language 
arts teachers were mandated by their districts to use CIMS-CA, 
but not exclusively. Actual usage in these schools was uneven, 
with some teachers using mostly CIMS-CA supplemented by basal 
readers, and others not using it at all. In all cases where 
school administrators strongly supported the program, usage 
increased. With only one CIMS-CA staff development manager for 
junior high schools in each district, monitoring curriculum use 
became the task of the school administrators. This, coupled with 
district budget cuts reducing the number of workshops and 
materials, frustrated staff developers in their efforts to assure 
a high standard of CIMS-CA usage. All CIMS-CA staff developers 
suggested that more project staff be hired. Some thought that 
teachers should be selectively mandated to help reduce possible 
misuse of the curriculum, which can result from inadequate 
training and support. 
DRAMA COMPONENT OVERVIEW 

In the fourth year of the drama component, approximately 
7,000 children participated, about the same number as last year. 
Teachers from grades four through eight were invited to 
participate. As with prior years, drama component activities 
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took place in conjunction with the Theater for a New Audience and 
vith the Equity Group, whose services were contracted through the 
Board of Education. 

Goals for the drana component for 1988-89 remained the same 
as last year: 

• to introduce the world of William Shakespeare; 

• to increase students* understanding and appreciation of the 
theater ; 

• to provide children with an opportunity to view a 
professionally produced play? 

• to encourage creative expression; and 

• to allow students to perform for an audience. 

Macbeth was the play chosen for this year's drama project. 
The drama component was divided into three phases, which included 
preparation, a trip to a live performance, and schoolwide 
performances, in previous years there had been an annual 
festival in Damrosch Park in Manhattan, but this was impossible 
this year because of site construction. 

The planning phase took place during a Christmas break 
workshop. Twelve teachers met to prepare grade appropriate 
lessons to be used with an already prepared abridged edition of 
the play, Macbeth. All teachers participating in the drama 
component were to receive one copy of the abridged play and 
accompanying lessons prepared during the workshop. Because 
listening, speaking, and appreciation of the play were the 
educational goals for this part of the CIMS-CA program, it was 
once again emphasized to teachers that students were not to read 

13 



24 



the play. Although there had been some confusion about these 
goals during the initial years of the component, there seemed to 
be a general understanding and acseptan<je of them for the 1988-89 
school year. 

The planning stage was continued in the bi-monthly and monthl., 
workshops for CIMS-CA teacher«s, \diere teaching strategies were 
developed and refined. In this way, teachers became involved in 
planning for their individual classrf»oms and the specific needs 
of their students. 

Artists from the Theater for a New Audience came to each 
school to conduct workshops for teachers. They visited 
classrooms and stimulated student appreciation for the play. As 
they had last year, staff developers, curriculum developers and 
artists prepared teachers for the project by preparing lessons 
about Shakespeare * s use of language; preparing lessons about the 
historical background of the play? indicating how prepared 
lessons could be accommodated to individual class nee :1s ; ard 
designing and acquainting teachers with games geared for teaching 
students how plays are produced. 

The second phase of the drama component was the trip to the 
theater to see Macbeth. For the most part, this phase of the 
project went very well. There seemed to be a good deal of 
improvement over last year, although some districts still 
experienced difficulty coordinating bus transportation for the 
students. Project staff, teachers, administrators, and parents 
helped with organizational tasks, and the vast majority of 
students did get to see the play. 
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The third phase was the classroom and schoolwide 
performemces o£ various aspects of Macbeth . Artists from the 
Theater for a New Audience and the Equity Group returned to the 
schools, and worked with teachers in preparation of performances. 
In many schools, parents helped vi^ preparations and 
performances. Most chool administrators accommodated the needs 
of drama component paxrticipants enthusiastically and efficiently. 

The elimination of the trip to Damrosch Park took some 
strain off districts' resources to provide transportation, and 
focused efforts to transport children to the theater performance 
were largely successful. 
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III. REACTIONS TO AND PERCEPTIONS OF 
THE INSTRUCTIONAL COMPONENT 

STAFF PERCEPTIONS OF C IMS-CA'S ADAPTABILITY AND EFFECTIVENESS 

Toward the end of the 1988-89 school year, teachers 
participating in CIMS-CA workshops wer© ^iven a questionnaire on 
the curriculum's effectiveness and its adaptability to students' 
individual needs. Teachers were asked if the curriculum promoted 
the integration of listening, speaking, reading, and writing 
skills, which is the major goal of the project. In addition, 
teachers were asked to rate the curriculum's appropriateness to 
students' interests; its adaptability to students' skill levels; 
and the usefulness of projec' developed resource books. Table 1 
illustrates the mean ratings in these four areas for teachers 
from all CIMS-CA districts, except C.S.D. 11. Teachers' 
^-esponses from C.S.D. 11 were analyzed separately because they 
had been trained by school -based supervisors, and not directly by 
CIMS-CA project staff like other CIMS-CA C.S.D.s. 

Although all four mean ratings {see Table 1) for usefulness 
and appropriateness of the CIMS-CA curriculum were quite 
favorable, teachers gave the highest rating to the curriculum's 
integration of listening, speaking, reading, and writing skills. 
Elementary school teachers tended to view the curriculum as more 
integrated than junior high school teachers did, but the 
difference was only slight, and not significant. Similarly, for 
the three remaining variables — appropriateness of curriculum to 
interests of students, adaptability of curriculum to children 
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TABLE 1 



Teachers* Ferceptions* of Usefulness and Appropriateness 
of the CIHS-CA Curriculum for C.S.D.s 3, 8, 9, 15, 17, and 30, 

(N=153) 



Variable Mean S.D. 



1. Curriculum promotes 
integrated use of 
listening, speaking, 

reading, and writing. 1.2 0.5 



2 . Appropriateness 
of curriculum to 

interests of students. 1.6 0.6 



3. Adaptability of 
cuTirjulum for children 
with different skills 

levels, 1.7 0.7 



4. Usefulness of 
resource books in 
implementing the 

curriculum. 1.7 1.1 



'All questions were rated using a four-point scale with 
l=highest rating and 4=lowest rating. 

Resource books used in all grades except kindergarten. 

• Teachers gave the highest rating to curriculim's integration 
of listening, speaking, reading, and writing skills. 
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with different skills levels, and usefulness of resource books in 
implementing the curriculum — there was no significant difference 
between elementary and junior high school teacher ratings. 
RllACTIONS T O STAFF DEVEIX^I^IENT ACTIVITIEg 

All CIMS-CA teachers taking part in after-school wor3cshops, 
including those in C.S.D. 11, were asked to rate the value of 
various CIMS-CA activities. Table 2 illustrates the outcome. 

For teachers in C.S.D. 11, demonstration lessons, individual 
conferences, and classroora observations were found to be the most 
valuable CIMS-CA activities. Teachers in all other C.S.D.s 
thought that after-school workshops, Shakespeare workshops, and 
demonstration lessons were of most value. Teachers in C.S.D. 11 
rated CIMS-CA activities slightly more favorf bly than those in 
other C.S.D.s, but without statistical significance. No 
discemable pattern emerged when responses were analyzed by 
grade . 

CURRENT AN D FUTURE USE OF CURRICUUJM 

Teachers from all C.S.D.s offered the following information: 
• Use of curriculum. All C.S.D.s except lis 70 percent used 
CIMS-CA curriculum as a supplement, 25 percent used it as 
primary curriculum and 4 percent used it as the only 
curriculum; C.S.D. 11: 72 percent used it as a supplement, 
21 percent as the primary curriculum, and 6 percent as the 
only curriculum. 
. Reported problems . All C.S.D.s except 11: 80 percent said 
there were no major problems; C.S.D. 11: 63 percent said 
there were no major problems. 
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TABLE 2 

Teachers Ratings* of Value of CIMS-CA Activities 



Variable 


District 11 
Mean S.D. 

(N=47) 


Other 
C.S.D. 
Mean 
(N«153) 


CIMS-CA 
S.D. 


1. After school, 
monthly or bimonthly 
staff development 
workshops . 2.0 


1.1 


1.4 


0.7 


2 . Shakespeare 
workshops . 


2.3 


1.5 


1.8 


1.0 


3 . Classroom 
observation. 


1.7 


0.9 


2.1 


0.9 


4 . Demonstration 
lessons . 


1.3 


0.7 


1.8 


1.0 


5. Faculty 
conferences. 


2.1 


1.0 


3.2 


0.1 


6. Grade conferences. 2.3 


1.2 


3.1 


1 • U 


7 . Individual 
conferences * 


1.5 


0.8 


2.0 


1.1 


8. Overall mean. 


1.9 




2.2 




*A11 questions were 
rating and 4=:lowe5t 


rated using 
rating. 


a four-point 


scale 


with l=highest 



• For teachers in C.S.D. 11, demonstration lessons, individual 
conferences, and classroom observations were found to be the 
most valuable CIM3-CA activities. 



• For teachers in all other CIMS-CA C.S.D.s, after-school 
workshops, Shakespeare workshops, and demonstration lessons 
were viewed as most valuable. 

• Teachers in C.S.D. 11 rated these CIMS-CA activities 
slightly more favorably than those in other C.S.D.s. 
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• Future use . All C.S.D.s except 11: 91 percent said they 
would like to use it next yean C.S.D. 11: information was 
not available. 

• Assistance . All C.S.D.s except 11: 89 percent said they had 
sufficient assistance in use of curriculum? C.S.D. 11: 95 
percent stated they had sufficient assistance. 

. Helpfulness of training. All C.S.D.S except 11: 95 percent 
stated that cms-CA training was either very helpful or 
r»oderately helpful i C.S.D. 11: 96 percent said the training 
was either very helpful or moderately helpful. 

PERCEPTIONS OF THE PRO GRAM ' S MAJOR STI^WgTHg A ^P WEMyE^g^S 

The following perceptions of program strengths and 
weaknesses were inferred from responses to open-ended questions 
on survey forms. These responses do not necessarily represent 
the opinions of the majority, since all respondents did not 
complete this section of the surveys. They are included because 
they offer valuable insights regarding program strengths and 
weaknesses that would be lost if only quantitative data were 
used. As with the quantitative data presented above, C.S.D. 11 
responses will be analyzed separately. 

Perceptio n s of Strengths of Teachers from CtStPt^ ?t 9, l^. 
17, and 30 

Teachers thought the major strength of the curriculum was 
the integration of reading and language arts with writing (N=22). 
Additionally, they cited the large variety of activiti'i^s included 
in the development of lessons (N=13). The interest-provoking 
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the&iatic organization of the curriculum Siaterials (N«ll) and the 
use of .authentic literature (N-'S) were also atentioned as 
ia^rtant strengths. 

Teachers appreciated the generous distribution of materials, 
texts, and books for cliissroos use (N=ll) . clear lesson plans 
and lesson manuals suitable for mc^ification (N»26} were noted as 
strcsngths. Teachers enjoyed sharing their ideas with other 
teachers in workshops (N«18) and were pleased with the degree of 
availability of staff developers. 

Teachers also thought the progra® was flexible enough to 
adjust activities to student needs (N«7), and that students 
learned a great deal from the activities (N=17) . Most 
impozrtantly, teachers seemed to think the progr€ua had succeeded 
in ^stimulating students to read and write about literature, and 
to participate in class activities (N«15) . 
Perceptions of strengths bv CIMS-CA Program Staff 

For the past several years, staff development managers have 
expressed extremely positive attitudes toward the program. The 
1988-89 school year was no exception. As in previous years, they 
placed a high value on training workshops, classroom visits, and 
the exchange of ideas among teachers and administrators. Working 
with new teachers was seen as an additional way of helping create 
a positive, successful learning atmosphere in th\^ schools. The 
teacher-developed curriculum? lessons based on the latest 
research; and increased use of the curriculum by English as a 
Second Language, special education, and bi-lingual teachers, were 
viewed as indicators of a successful and effective program. 
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jPereeptio n s of WeaknftssftS bv Teacherg ivm gtStPtS ?« 9« l^s . 
17. aCTd 30 

The two Bost f xrequently cited weaknesses were that the 
curriculism laaterials were not suited for slow learners (N=10) , 
and that activities were too long (N-5) . Other than these, there 
was little consensus on wealoiesses. 
Percept ion ff ^ yff im passes bv CIMS-CA Project Staff 

The three weaknesses naaed by CIMS-CA staff development 
managers were: 1) the ratio between project staff and teachers 
was too high (ie. more staff-development managers should be 
hired) i 2) the syatem in which CIMS-CA project staff trained 
teacher trainers in C.S.D. 11 to conduct CIMS-CA workshops 
reduced the project staff's ability to ensure proper use of the 
curriculum? and 3) mandating the project for all teachers in a 
district brought unwilling teachers into the project. 

STAFF SUGG ESTIONS FOR CHANGE 

Teachers from C.S.D. s 3. 8. 9. Ig, 17. and 3Q 

Suggestions for change included the elimination of certain 
selections in the curriculum — "Lunar lAire" was mentioned (N=12), 
the addition of more workshops (N=4), and the opportunity for 
more staff to participate in the project (N«4) . It should be 
noted that these suggestions were constructive and focused on 
teachers* wishes to get more involved with the project. 

Teachers from C.S.D. 11 suggested that they be given an 
opportunity to meet with CIMS-CA project staff CN=4), and that 



materials be distributed more quickly (Nb3). As vith teachers 
£r^ other C.S.D.s, these suggestions indicated a desire for 2:;ore 
contact with the project. 

Staff developi&ent managers thought that their numbers should 
be increased, and that they should have a more direct role in 
training teachers in C.S.D.s idiere the program is mandated, 
notably, C S.D. 11. They also thought mandating should be 
selective within schools so that fewer unwilling teachers would 
enter the program. 

The project research manager looked forward tc having the 
assessment component used by all C.S.D.s, vith the now-available 
software for F.C.s. 

PERCEPTIONS OF THE D RAMA COMPONENT 

Project staff and teachers were asked to rate the value of 
the drama component. Teachers answered questions that were a 
part of the CIMS-CA teacher survey, while project staff responded 
to open-ended questions and face-to-face interviews. Students 
were also asked to provide their opinions by writing essays in 
class. 

Project Staff's Perceptions of the D rama Component 

This year, as in the past three years, the project staff 

gave the drama component its highest ratings. This consensus 

im ded all staff -development managers, the research 

coordinator, and the project director. 

The large number of students participating in the project 

necessitated a great deal of organizational skill on the part of 
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the project staff. Some staff sestbers stated that they needed 
additional help, hut veteran staff agreed that this year's drama 
component was better organized ^an those of previous years. 
Eliminating the Dasrosch Park event was seen as helpful in that 
resources were better distributed to the other fdiases. 

Transportation continued to be a problematic issue in some 
districts, but not all. In those districts where transportation 
of students to the theater in Manhattan was less than optimum, 
project staff saw this as a problem with the district office, not 
with their own planning. In fact, transportation is the 
districts' responsibility, not that of the project. 

In some districts, staff developers thought school 
administrators could have been more helpful in in^lementing the 
schoolwide performances. However, they noted that the large 
number of student participants may have placed a strain on these 
schools. In other districts, administrator and parent 
involvement helped the schoolwide performances to be very 
successful. 

Teachers' Perceptions of the Drama Component 

The large majority of teachers in our survey sample rated 
the drama component as very effective. Teachers were sked to 
rate five aspects of the drama components 1) usefulness of 
training materials? 2) usefulness of preplay workshops? 3) 
usefulness of postplay workshops; 4) effectiveness of artists' 
visits? and 5) overall program effectiveness in enriching 
children's understanding of drama. In every category, the 
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najority of teachers surveyed gave the draroa component the 



highest ratings (see Table 3) . 

Students' Perceptions of the i^aaa Coi^p^n^nt 

For the past three years, students* perceptions have been 
included in the report, students were asked to vrite an essay 
reflecting their thoughts about the project. The following 
student statements, all positive, may or say not be 
representative or student opinion, but they have been included as 
anecdotal information illustrating how some students benefitted 
from the drama component: 

• "When we first got there (the theater) I thought it would be 
boring. But as I listened and watched the characters, it 
made sense to me. I liked the way they said their parts and 
the way they moved and acted on stage." (fourth grade) 

• "I think the play was superb and the acting was great. If I 
had a choice of a play I would pick llasfegj^. I would give 
the play the best mark that a person can give a play." 
(si>cth grade) ' 

• "I thought the play was fantastic. Although the words were 
hard to understand, I paid close attention to them so I 
could figure out what was going on. I am looking forward to 
performing a ten-minute . scene from Macbeth in the spring." 
(seventh grade) 

• «It»s so exciting to see a play close up like that. All the 
scenes were wonderful. I also love the light settings, and 
everything else." (seventh grade) 

• "My class really loved the play. I hoped we could see it 
again. Although we are not going to be able to see it 
again, my classmates cannot stop thinking or talking about 
the play Macbett^ ." (sixth grade). 




TABLE 3 



Teachers* Ratings' of CIMS-CA Dra&a Component, 
All C.S.D.S, Except C.S.D.ll, {N«73) 



Variable Mean S.D. 



1. Usefulness of 
preplay workshops in 

preparing for play. 1.3 0.6 



2. Usefulness of 
postplay workshops in 

preparing for perf orrnances . 1.4 0.6 



3. Effectiveness of 
artists* visits in helping 
children prepare for 

performance. 1.2 0.5 



4. Overall effectiveness 
of draata component in 
enrichment of students* 

understanding of drama. 1.3 0.5 



"All questions were rated using a four-point scale with l=highest 
rating, and 4=lowest rating. 

• Teachers rated the drama component as useful and effective. 
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IV. STUDENT OUTCOMES 



METHODO^)Glf 

The ispact of the CIMS-CA program on student achievement was 
determined by examining the change in participating students* 
reading and writing performance between 1988 and 1989. Students 
participating in the p^^itam %fere identified by those teachers 
\rtio had used the CIMS-CA curriculum during the 1988-89 school 
year. Pretest and posttest comparisons of student achievement by 
district and by grade were made on ^itandardized reading 
achievement tests and holistically scored writing samples. 
Reading analyses included all CIMS-CA participants, while the 
writing analyses were conducted on a stratified sample of 
participants . 
Reading 

In order to measure the impact of the CIMS-CA curriculum on 
students* reading achievement, CIMS-CA students* scores on 
citywide standardized reading tests administered in spring 1988 
were compared to their scores on reading tests administered in 
spring 1989. Students in grades three through eight were 
administered the Degrees of Reading Power (D.R.P.) test series in 
1989. For third grade students, 1989 reading scores on the 
D.R.P. were converted to comparable test scores on the 
Metropolitan Achievement Test (M.A.T.), which was administered to 
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second grade students in 1988** For grades four through eight, 
no conversion was needed since the D.R.P. was administered in 
both 1988 and 1989. CIMS-CA students with cos^lete test 
inforaation in reading were from C.S.D.s, 3, 8, 9, 11, 15, 17, 
24, and 30 {N»8,208} . 

Analyses of reading achievestent were performed on students* 
normal curve equivalent scores (N.C.E.s)** in grade three, and on 
students* D.R.p. units*** in grades four through eight. Reading 
improvement was operationally defined as statistically 
significant increases in students* reading scores from 1988 to 
1989. Correlated t-tests were used to determine whether the 
reading test scores of CXMS-CA students in grades three through 
eight were significantly higher in 1989 than in 1988. 
Statistical significance indicates whether the changes in 
achievement are real or occur by chance. However, statistical 
significance can be exaggerated by large sample size or depressed 



*Table5 for converting test scores were generated with the 
equipercentile method in which frequency distributions of scores 
on both tests were equated. 

**N.C,E. scores are similar to percentile ranks but, unlike 
percentile ranks, are based on an equal -interval scale. N.C.E. 
scores are based on a scale ranging from one to 99 with a mean 
of 50 and a standard deviation of approximately 21. Because 
N.CE. scores are equally spaced, arithmetic and statistical 
calculations such as averages are meaningful,* in addition, 
comparisons of N.CE. scores may be made across different 
achievement tests. 

***In the D.R.P. program, readability is measured in D.R.P. 
units. D.R.P. units form a scale of prose difficulty or 
readability. Theoretically, the scale ranges from 0 to lOO 
units. In practice, commonly encountered English text mns from 
about 30 D.R.P. units to about 85 D.R.P. units. The higher the 
D.R.P. unit value, the more difficult the text. 
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by small sample size. Furthermore, statistical significance does 
not address the issue of whether the a^ievessent changes are 
important to the students* educational develo^ent. Thus, an 
effect size (E.S.)* is reported for each significant difference 
to indicate the educational meaningfulness of each mean gain or 
loss, independent of the sample size. 
Writing 

To assess CIMS-CA students' improvement in %rriting skills 
from 1986 to 1989, samples of students* vrriting wer^ collected 
both during fall 1988 and spring 1989. A stratified sample of 
matched writing samples (27%) from 2,191 students Including both 
fall 1988 and spring 1989 writing samples for each student were 
selected oy grade and district from the total number of writing 
samples collected. These samples were holistically scored, and 
correlated t-tests were used to determine statistical 
significance in the difference between the fall 1988 and spring 
1989 scores. 

Three holistic scoring schemes were developed for scoring 
writing samples for the following grade clusters: kindergarten 
second grade, third grade to sixth grade, and seventh and eighth 
grades. To assess the reliability of the holistic scoring 
schemes, pairs of scorers were asked to rate the same writing 

*The effect size, developed by Jacob Cohen, is the ratio of the 
mean gain to the standard deviation of the gain, ^is ratio 
provides an index of improvement in standard deviation units 
irrespective of the size of the sample. According to Cohen, 0.2 
is a small E.S., 0.5 is a moderate E.S., and 0.8 is a large E.S. 
Only effect sizes of 0.8 and above are considered to be 
educationally meaningful, reflecting the importance of the gains 
to the students' educational development. 
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samples within each scoring scheme. The percentage of agreeiaent 
for the kindergarten to second grade cluster was 82.8; the 
percentage of agreexaent for the third gr^de to sixth grade 
cluster was 76.3, and the percentage of agreement for the seventh 
and eighth grade cluster was 72.7. These percentages of 
agreement indicate that the interrupter reliability is 
satisfactory. 

C.S.D.s 3, 8, 9, 11, 15, 17, and 30 were included in the 
analysis of writing achievement. However , writing samples were 
not collected for all grade clusters in each C.S.D. Statistical 
significance of changes from 1988 to 1989 are reported for the 
grade clusters in each district. Ag^ln, an E.S. is reported for 
each significant difference as an indicator of the educational 
meaningfulness of the mean gain or loss. 
RESULTS OF READING ACHIEVEMENT TESTS 

Reading achievement results for grade three through eight 
are presented in Tables 4 through 14. Results for third grade 
students are presented first, followed by the results for grades 
four through eight. 
Third Grade Reading ' 

Table 4 presents the result from the correlated t-test 
analyses of third grade students' reading achievement, overall, 
there was a significant mean loss of 4.8 N.C.E. points 
(S.D.«15.3) in students » reading achievement in C.S.D. 3, 11, 15, 

Tiote that only N.C.E. reading scores were available for third 
grade comparison because of the need to equate scores obtained 
for the 1989 D.R.P. reading test and for the 1988 M.A.T. reading 
test. Note also that a new equating table was used this year, 
since a new norm was established for the D.R.P. test. 
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CIMS-CA R£jlU>ING 1988-89 
Pre- and Postcomparison of Reading scores for Grade Three' 







Pretest 


Posttest 


Difference,. 




C.S.D. 


N 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 




03 


71 


34.2 


16.6 


30.7 


14.1 


-3.5 


13.1 


0.3 


11 


1,361 


57.3 


19.2 


53.1 


21.0 


-4.2 


15.5 


0.3 


15 


77 


56.3 


18.7 


46.5 


18.3 


-9.8 


14.9 


0.7 


30 


204 


48.7 


16.1 


41.3 


15.6 


-7.4 


13.7 


0.5 


Total 


1,713 


55.3 


19.4 


50.5 


20.9 


-4.8 


15.3 


0.3 



"Grade 3 was evaluated using 1988 M.A.T. N.C.E. scores and converted 1989 
D.R.P. N.C.E. scores. 

'^hese mean differences were significant at p<.05 level. 



• There was an overall statistically significant mean loss of 4 . 8 
N.C.E. points. This mean difference represented a low effect 
size. 



31 

42 



and 30. However, the overall laean loss only represented a small 
effect size. Although the xoean reading losses for individual 
districts were all statistically significant, the mean losses for 
C.S.D. 3 (3.5 N.C.E. points, S.D. - 13.1) and C.S.D. 11 (4,2 
N.C.E. points, S.D. -'IS. 5) were both low in effect size while the 
mean losses for C.S.D. 15 (9.8 N.C.E. points, S.D. » 14.9) and 
C.S.D. 30 (7.4 N.C.E. points, S.D. = 13.7) were moderate in 
effect size. 

(grades Fffyr Through Eight Reading Aehleveaenfe 

A comparison of CIMS-CA students' mean gains in reading 
scores between spring 1988 and spring 1989 is presented by grade 
in Table 5. This table shows that there was an overall 
statistically significant mean increase of 7.2 D.R.P. units 
(S.D.-8.6) from 1988 to 1989. This represented a large E.S. 
Grades four, five, and eight showed generally large E.S.s for 
their mean gains, whereas the differences for grades six and 
seven represented moderate E.S.s. 

Reading achievement results for grade four through eight are 
also presented by districts in Table 6. Two-thirds of the sample 
were frca C.S.D. 11, in which an overall statistically 
significant and educationally meaningful mean gain of 7.9 D.R.P. 
units (s.D.«8.8) was achieved, other C.S.D. s that achieved 
statistically significant and educationally meaningful mean gains 
on their reading achievements were C«S.D,s 3, 24, and 30. The 
remaining C.S.D.s— 8, 9, 15, and 17 — achieved statistically 
significant mean gains that represented moderate effect sizes. 
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TABLE 5 



CIKS-CA READING 1988-89 

Pre- and Postcoffiparison cf D.R.P. Reading Scores for 

All Districts by Grade 







Pretest 


Posttest 


Difference- 




Grade 


N 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


£ • S • 


4 


1,638 


40.8 


14.7 


49.6 


12.2 


8.8 


9.4 


0.9 


5 


1,320 


47.4 


12.3 


57.5 


11.5 


10.2 


8.6 


1.2 


6 


1,758 


54.6 


11.5 


59.9 


11.5 


5.3 


8.1 


0.7 


7 


946 


59.2 


13.3 


64.5 


11.4 


5.3 


7.8 


0.7 


8 


833 


63.2 


10.2 


68.4 


11.6 


5.1 


6.7 


0.6 


Total 


6,495 


51.4 


14.9 


58.6 


13.2 


7.2 


8.6 


0.8 



"These mean differences were statistically significant at p<.05 level. 



• There was an overall statistically significant mean gain of 7.2 
D.R.P. units. This gain was educationally meaningful. 

• Mean gains for grades four, five, and eight represented large 
effect sizes and were educationally meaningful. 

• Mean gains for grades six and seven represented moderate effect 
sizes. 
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TABI£ 6 

cms-ch wa>im 1988-89 

Pre- and Postcoi^rison of D.R.P. Reading Scores for 

Grades Four through Eight* 







Pretest 


Posttest 


Qifference^ 




C.S.D. 


N 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


E.S 


03 


510 


42.0 


14.9 


50.7 


12.6 


8.7 


8.0 


1.1 


08 


247 


64.9 


10.0 


69.7 


11.0 


4.8 


6.7 


0.7 


09 


438 


54.6 


13.7 


58.6 


13.9 


4.1 


7.8 




11 


4,169 


50.5 


14.3 


58.4 


12.6 


7.9 


8.8 




15 


431 


55.7 


15.2 


61.7 


14.4 


6.0 


9.4 


0.6 


17 


501 


60.7 


8.8 


64.2 


9.5 


3.5 


6.8 


0.5 


24 


24 


45.3 


6.2 


55.3 


7.9 


10.0 


6.6 


1.5 


30 


175 


38.3 


15.9 


47.1 


14.8 


8.8 


7.9 


1.1 


Total 


6,495 


51.4 


14.9 


58.6 


13.2 


7.2 


8.6 


0.8 


•Grades four through eight 
Midinstructional 


were 


evaluated using 1988 and 


1989 D.R.P. 





scores . 

**rhese mean differences were statistically significant at p<.05 xevel. 

• There was an overall statistically significant mean gain of 7.2 
D.R.P. units. This gain was educationally meaningful. 

• Mean gains for districts 3, 11, 24, and 30 represented large 
effect sizes and were educationally meaningful. 

• Mean gains for districts 8, 9, 15, and 17 represented moderate 
effect sizes. 
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The results for Individual grades within each district are 
presented in Tables 7 through 14. ^ese tables show the 
following: 

• C.S.D. 3 showed an overall statistically significant and 
educationally significant aean gain of 8.7 D.R.P. units 
(S.D.*8.0). Although all grades in C.s.D. 3 made 
significant and meaningful gains in reading, for grades four 
through eight there ai^ared to he a trend for lower grades 
to achieve higher mean gains than upper grades. 

• There was an overall statistically significant increase in 
reading scores of 4.6 D.R.P. units (S.D.>6.7) in C.S.D. 8. 
This represented a moderate E.S. Students in the two 
participating grades in this district, seven and eight, 
increased their scores significantly by 4.3 {S.D.»6.6}, and 
4.7 (S.D.«=6.7), respectively. These changes represented 
moderate E.S.s. 

• C.S.D. 9 showed an overall statistically significant mean 
gain in reading scores of 4.0 D.R.P. units (S.D.«=7.8). This 
mean gain represented a moderate E.S. Students in the three 
participating grades in this district, six, seven, and 
eight, increased their scores significantly by 3.4 
(S.D.-7.9), 4.3 (S.D.-8.0), and 5.4 D.R.P. units (S.D.«7.2) 
respectively. The mean gains for grades six and eight 
represented large E.S.s. The mean gain for grade seven 
represented a moderate E.S. 

• There was an overall statistically significant increase in 
reading scores of 7.9 D.R.P. units {S.D.«8.8) in C.S.D. 11. 
This gain represented a large E.S. Students in grades four 
through eight showed statistically significant increases in 
reading scores. These gains represented moderate to large 

• C.S.D. 15 showed an overall statistically significant mean 
gain in reading scores of 6.0 D.R.P. units (S.D.=9.4). This 
gain represented a moderate E.S. Students in grades five 
through eight made statistically significant increases in 
reading scores. These increases represented moderate to 
large E.S.s. 

• There was an overall statistically significant increase in 
reading scores of 3.5 D.R.P. tmits (S.D.«6.8) in C.S.D. 17. 
This gain represented a moderate E.S. Students in grades 
six, seven, and eight increased their reading scores 
significantly by 1.2 (S.D.-7.3), 6.2 (S.D.-5.5), and 4.9 
(S.D.=5.8), respectively. Mean gains for grades seven and 
eight represented large effect sizes. 
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TAB££ 7 



CIMS-CA READING 1988-89 

Pre- and Postc^oitparison of D.R.P Midinstructional 
Reading Scores for District 3 



GRADE 


N 


Pretest 


Posttest 


Difference* 


E.S. 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


4 


154 


27.4 


<^.2 


38.7 


9.6 


11.3 


7.3 


1.5 


5 


131 


42.5 


13.1 


52.6 


9.5 


10.1 


9.3 


1.1 


6 


126 


50.9 


11.1 


57.9 


10.1 


7.0 


7.2 


1.0 


7 


85 


52.6 


9.3 


57.9 


9.4 


5.3 


6.2 


0.9 


8 


14 


53.1 


6.9 


56. 


5.2 


3.3 


4.2 


0.8 


Total 


510 


42.0 


14.9 


50.7 


12.6 


8.7 


8.0 


1.1 



•These roean differences were significant at p<.05 level. 



• There was an overall statistically significant gain of 8.7 score 
points. This mean gain was educationally meaningful. 

• Grades four through eight showed positive mean gains. These 
were educationally oeaningfui and represented large effect 
sizes. 
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TABLE 8 

CIMS-CA READIKG 1988-89 

Pre* and Postcoi^rison of D.R.P. MidinstxuctionaX Reading 

Scores for District 8 







Pretest_ 


Posttest 


Difference" 




GRADE 


N 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


E.S. 


7 


73 


63.4 


8.7 


67.7 


7.6 


4.3 


6.6 


0.7 


8 


165 


66.5 


9.6 


71.2 


11.7 


4.7 


6.7 


0.7 


Total 


238 


65.6 


9.4 


70.1 


10.7 


4.6 


6.7 


0.7 



"These mean differences were significant at p<.05 level 



• There was an overall statistically significant gain of ^ .6 score 
points which was moderate in effect size. 
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TABLE 9 

Cn^-CA REJIDZHG 1988-89 

Pre- and PostooB^rison of D.R.P. Miainstructional Reading 

Scores for District 9 







Pretest 


I^sttest 


Difference" 




GRADE 


N 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


£.S. 


6 


206 


47.7 


8.5 


51.1 


8.7 


3.4 


7.9 


1.4 


7 


153 


57.0 


15.4 


61.3 


14.2 


4.3 


8.0 


0.5 


8 


79 


67.7 


10.1 


73,1 


10.6 


5.4 


7.2 


0.8 


Total 


438 


54.6 


13.7 


58.6 


13.9 


4.0 


7.8 


0.5 



•These nean differences were significant at p<.05 level. 



• There was an overall statistically significant gain of 4.0 score 
points. This mean gain was moderate in effect size. 

• Grades six, seven, and eight showed statistically significant 
positive mean gains. 

• The mean gain for grade six and eight represented large effect 
size. 



• The mean gain for grade seven represented a moderate effect 
size. 
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TABLE 10 

CIMS-CA READIH6 1988-89 

Pre- and Postcomparison of D.R.P. Midinstructional Reading 

Scores for District 11 







Pretest 


Posttest 


Difference' 




GRADE 


N 






n%pca4« 


S.D. 


Mean 


S.D. 


E.S. 


4 


1,322 


42.4 


14.1 


51.2 


11.3 


8.8 


9.7 


0.9 


5 


1,106 


47.8 


11.7 


58.0 


11.3 


10.2 


8.0 


1.3 


6 


1,106 


55.9 


11.3 


62.0 


11.3 


6.1 


8.1 


0.8 


7 


342 


60.8 


12.7 


66.1 


9.6 


5.3 


8.6 


0.6 


8 


293 


64.5 


9.6 


69.8 


11.1 


5.3 


7.1 


0.7 


Total 


4,169 


50.5 


14.3 


58.4 


12.6 


7.9 


8.8 


0,9 



"The mean differences were significant at p<.05 level. 



• There was an overall statistically significant gain of 7.9 score 
points. This nean gain was educationally meaningful. 

• All grades showed statistically significant mean gains. 

• Grades four through six showed educationally meaningful gains. 

• Mean gains for grades seven and eight represented a moderate 
effect size. 
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TABLE IX 

CIHS-CA READING 1988-89 

Pre- and Postcoaparison of D.R.P. *!idinstructional Reading 

Scores for District 15 



FT^t^t PPgttf»8t. Difference 

GRArr N Mean S.D. Mean S.D. Mean S.D. E.S. 



4 


27 


43.2 


16.3 


43.7 


13.8 


0.5 




7.9 0.1 


5 


71 


52.5 


14.0 


61.5 


14.5 


9.0* 


14.2 


0.6 


6 


57 


50.7 


11.3 


58.4 


9.7 


7.7* 


9.1 


0.8 


7 


174 


61.3 


16.3 


66.5 


13.6 


5.2" 


7.9 


0.7 


8 


102 


54.5 


11.7 


60.2 


13.7 


5.7* 


6.9 


0.8 


Total 


431 


55.7 


15.2 


61.7 


14.4 


6.0' 


9.4 


0.6 



*The mean differences were significant at p<.05 level. 



• There was an overall statistically significant gain of 6.0 score 
points. This mean gain represents a moderate effect size. 

• Grades five through eight showed statistically significant mean 
gains. * 

• Grades six and eight showed gains that represented large effect 
sizes. The gains in grades five and seven represented moderate 
effect sizes. 
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TABLE 12 



CIHS* CA READING 1988-89 

Pre- and Postccjaparison of D.R.P. Midinstructional Reading 

Scores for District 17 



GRADE 


N 


Pretest 


Pesttest 
Mean S.D. 


Oifference- 


E.S. 


Mean 


S.D. 


Mean 


S.D. 


€ 


224 


60.2 


10.3 


61.4 


10.6 


1.2 


7.3 


0.2 


7 


97 


59.5 


6.8 


€5.7 


7.4 


6.2 


5.5 


1.1 


8 


180 


62.0 


7.3 


66.9 


8.1 


4.9 


5.8 


0.8 


Total 


501 


60.7 


8.8 


64.2 


9.5 


3.5 


6.8 


0.5 



•The nean differences were significant at p<.05 level. 



• There was an overall statistically significant gain of 3.5 score 
points. This qain represents a moderate effect size. 

• Grades six through eight showed statistically significant mean 
gains . 

• Mean gains for grades seven and eight represented large effect 
sizes. 
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TABLE }3 



CIMS-CA READING X988-89 

Pre- and Postcomparison of D.R.P. Midinstructional Reading 

Scoxres for District 24 













Difference* 




GRADE 


N 


Mean S.D. 


Mean S.D. 


Mean S.D. 


E.S. 


7 


22 


44.9 5.8 


55.1 8.2 


10.2 6.6 


1.5 



■The mean difference was significant at p<.05 level. 

• Mean gain for grade seven was statistically significant and 
represented a large effect size. 
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TABLE 14 



CIHS-CA IffiADING 1988-89 

Pre- and Postcomparison of D.R.P. Midinstructional Reading 

Scores for District 30 



GRADE 


N 


Pretest 


Posttest 


Difference' 


E.S. 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


4 


132 


38.7 


17.2 


47.5 


16.1 


8.8 


8.1 


1.1 


5 


8 


24.0 


11.2 


38.4 


7.7 


14.4 


10.1 


1.4 


6 


35 


39.8 


8.8 


47.3 


9.4 


7.5 


6.3 


1.2 


Total 


175 


38.3 


15.9 


47.1 


14.8 


8.8 


;.9 


1.1 



•These mean differences were significant at p<.05 level. 



• There was an overall statistically significant gain of 8.8 score 
points. This mean gain was educationally meaningful. 

• All grades showed statistically significant mean gains that were 
also educationally meaningful. 
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• District 24 also showed a statistically significant and 
meaningful gain of 10.2 D.R.P. units (S.D.-6.6). However, 
only a small sample in grade seven is represented in this 
district. 

• C.S.D. 30 shoved an overall statistically significant mean 
gain in reading scores of 8.8 D.R.P. units (S.D.«7.9). This 
mean gain represented a large E.S. The three participating 
grades in thi«5 district increased their riding scores 
significantly. These changes all represented large E.s.s. 

Suromarv of Reading Achievement 

In general, all grade four through eight students in 
participating districts achieved overall statistically 
significant mean gains in reading when the 1988 reading scores 
were coappred to their 1989 reading scores. These gains 
generally represented moderate to large E.S.s and were 
consistently larger for grades four and five than for grades six 
through eight. By grade, mean gains were generally smaller for 
grades in which students performed above the 50th D.R.P. unit at 
pretest than for grades in which students were below the 50th 
D.R.P. unit at pretest. 

For grade three students in the program, there appeared to 
be a consistent drop in N.C.E. reading scores for the four 
participating districts. Note that N.C.E. scores compare 
students' reading achievement in the program to other students 
across the country and may not measure individual student's real 
reading gain from one year to another. 

RESULTS OF WRITING AC HIEVEMENT TRSTfi 

Table 15 reports the results of the overall comparisons of 
fall 1988 and spring 1989 writing achievement tests for students 
in kindergarten through grade eight. The writing scores are 
presented in three grade clusters; kindergarten to grade two, 
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TABLE 15 



Analysis of 1988-89 CIMS-CA Writing Samples 
for all c.S.D. by Grade Clusters 



Grades 



N 



Pretest 
Mean S.D. 



Posttest 
Mean S.D. 



Mean S.D. 



Effect 
Size 



K-2 


710 


3.7 


2.0 


5.1 


1.8 


1.4 


1.5 


0.9 


3-6 


931 


3.2 


0.9 


3.9 


1.1 


0.7 


1.0 


0.7 


7-8 


550 


3.7 


1.0 


3.8 


1.0 


0.1 


1.0 


0.1 


Total 


2,191 


3.5 


1.4 


4.3 


1.5 


0.8 


1.3 


0.6 



•Mean differences were statistically «?jignif icant at p<.05 level. 

• There was an overall statistically significant mean gain of 0.8 
points. This mean gain was moderate in effect size. 

• Grade Kindergarten through two showed educationally meaningful 
gains. 

• Mean gain for grades three through six represented a moderate 
effect sizes. 
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grades three through six, and grade seven and eight. This table 
shows that there was an overall statistically significant mean 
increase in cms-Ch students* writing scores of 0.8 points 
(S.D.Bl.3) from fall 1988 to spring 1989. This mean increase 
represented a moderate Additionally, the lower primary 

grade cluster (K-2) achieved a mean writing gain (1.4 points, 
S.d.bI.5} that was large in E.S., the grades three through six 
cluster achieved a mean gain (0.7 points, S.D.«1.0} that was 
moderate in E.S., i^ile the mean gain for grades seven and eight 
(0.1 points, S.D.si.O) was low in E.S. 

Tables 16 to 22 present similar pre- and postcomparisons for 
individual grade clusters within each of the following districts: 
C.S.D.s 3, 8, 9, 11, 15, 17, and 30. (Pre- and postcomparisons 
for individual grades within each district are presented in 
Appendices Al through AS.) 

Tables 16 tl^^rough 22 show the following findings on 
students' writing achievement! 

• CIMS-CA students in C.S.D. 3 showed an overall statistically 
significant mean gains of 1.2 points (S.D.sl.5). This gain 
represented a large E.S. Students in the Kindergarten 
through second grade cluster through achieved educationally 
meaningful gains while students in the 7 and 8 grade cluster 
did not. 

• Only students in the 7 and 8 grade cluster was represented 
in C.S.D. 8. These students did not achieve a statistically 
significant mean gain (0.1 point, S.D.^l.O). 

• Students in C.S.D. 9 showed an overall mean gain of 0.2 
points (s.D.el.l) but this gain was not statistically 
significant. 

• An overall statistically significant and educationally 
meaningful mean writing gain was achieved by students in 
C.S.D. 11 (0.9 points, .S.D.>-1.2). The mean gains for 
students in the Kindergarten through second and third 
through sixth grade clusters represented moderate to large 
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TABLE 16 



Analysis of 1988-89 CIMS-CA Writing Sasqples 
for C.S.D. 3 by Grade Clusters 



Pretest Pggttest Piff^r^n?? Effect 

Grades N Mean S.D. Mean S.D. Mean S.D. Size 



K-2 


61 


2.2 


1.4 


3.7 


1.9 


1.5« 


1.6 


0.9 


7-8 


13 


3.1 


O.S 


3.2 


0.4 


0.1 


1.0 


0.1 


Total 


74 


2.4 


1.3 


3.6 


1.7 


1.2* 


1.6 


0.8 



^ean differences were statistically significant at p<.05 level. 



• There was an overall statistically significant mean difference 
which was educationally meaningful. 

• Grade Kindergarten through two showed educationally meaningful 
gains . 

• Mean gain for grades seven and eight represented a small effect 
sizes. 
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TABLE 17 

Analysis of 1988-69 CIMS-CA Writing Sai^les 
for C.S.D. 8 by Grade Clusters 









Pretest 


Posttest 


Difference 


Effect 


Grades 


N 


Mean S.D. 


Mean S.D. 


Mean S.D. 


Size 


7-8 


97 


4.0 0.9 


4.1 1.0 


0.1 1.0 


0.1 



• There was no statistically significant mean difference in pre- 
and postwriting scores in C.S.D. 8. 
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TABI£ 18 



Analysis of 1988-89 cms-CA Writing Samples 
for C.S.D. 9 by Grade Clusters 



Pretest PggUggt Piffgr^ffg Effect 

Grades N Mean S.D. Mean S.D. Mean S.D. Size 

3-6 51 3.5 0.8 4.0 0.9 0.5' 0.9 0.6 

7-8 103 3.9 1.1 4.0 1.0 0.1 .2 0.1 

Total 154 3.8 1.0 4.0 0.9 0.2 1.1 0.2 

"Mean differences were statistically significant at p<.05 level. 

• There was no overall statistically significant mean difference. 

• Grades three through six showed a statistically significant mean 
gain. This gain represented a moderate effect size. 
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TABLE 19 

Analysis of 1988-89 CIl^-CA Writing Sas^les 
for C.S.D. 11 by Grade Clusters 



ftr^tggt Posttest Difference Effect 

H Kean S.D. Mean S.D. Mean* sTd. Siz^ 



K-2 


507 


4.0 


1.9 


5.3 


1.7 


1.3 


1.4 


0.9 


3-6 


670 


3.2 


0.9 


3.9 


1.2 


0.7 


1.0 


0.7 


7-8 


154 


3.7 


0.8 


3.9 


1.0 


0.2 


1.0 


0.2 


Total 


l,3ri 


3.5 


1.4 


4.4 


1.5 


0.9 


1.2 


0.8 



•Mean differences were statistically significant at p<.05 level . 

• There was no overall statistically significant gain of 0.9 
points. This mean gain was educationally meaningful. 

• All grade clusters showed statistically significant mean gains. 

• Grades Kindergarten through two showed educationally meaningful 
gains. 

• Mean gains foi grades three through six represented a moderate 
effect size. 
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TABLE 20 



iVnalysis of 1988-89 CIMS-CA Writing Samples 
for C.S.D. 15 by Grade Clusters 



Grades 


N 


Pretest 


Posttest 




Effect 
Size 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


K-2 


52 


2.8 


1.7 


5.7 


1.1 


2.9' 


1.6 


1.8 


3-6 


95 


3.6 


0.9 


4.1 


0.8 


0.5* 


1.0 


0.5 


7-8 


73 


2.6 


0.9 


2.8 


0.8 


0.2 


0.9 


0.2 


Total 


220 


3.2 


1.2 


4.1 


1.4 


l.O* 


1.6 


0.6 



"Mean differences were statistically significant at p<.05 level. 
• There was an overall statistically significant gain of 1.0 



point. 

• Grades Kindergarten through two showed educationally meaningful 
gains . 

• Mean gain for grades three through six represented a moderate 
effect size. 
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TABZ£ 21 

ilnalysis of 1988-89 CZHS-CA Writing Sasples 
for C.S.D. 17 by Grade Clusters 



Grades 


N 


Pretest 


Poal 
Mean 


^test 
S.D. 


Difference 


Effect 
Size 


Kean S.D. 


Mean 


S.D. 


3-6 


38 


2.9 0.7 


3.8 


0.9 


0.9' 


1.0 


0.9 


7-8 


98 


4.0 0.9 


3.8 


0.9 


-0.2 


1.0 


0.2 


Total 


136 


3.7 0.9 


3.8 


0.9 


0.1 


1.1 


0.1 



■^lean differences were statistically significant at p<.05 level. 

• There was an overall statistically significant aean difference. 

• Grades three through six showed a statistically significant mean 
gain, and it was educationally meaningful. 
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TABLE 22 



Analysis of 1988-89 CIMS-CA Writing Saioples 
for C.S.D. 30 by Grade Clusters 



Pretest Posttest Pifggrengg Effect 

Grades N Mean S.D. Mean S.D. Mean* S.D. Size 



K-2 


90 


3.6 


2.1 


4.9 


2.1 


1.3 


1.3 


1.0 


7-8 


77 


2.6 


0.8 


3.3 


0.9 


0.7 


0.9 


0.8 


Total 


167 


3.1 


1.7 


4.2 


1.8 


1.1 


1.1 


1.0 



^ean differences were statistically significant at p<.05 level. 

• There was an overall statistically significant gain of 1.1 
points. This mean gain was educationally meaningful. 

• All grade clusters showed statistically significant and 
educationally meaningful gains. 
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E.S.8, vhile the mean gain chieved by students in the seven 
to eight grade cluster was low in 5.S. 

• Students in C.S.D. 15 also achieved an overall statistically 
significant mean gain in their writing scores (l.o points, 
S.D.»1.6). The ^an gain represented a moderate E.S. 
Students in the Kindergarten through second grade cluster 
achieved writing gains that were educationally meaningful 
while students in higher grades did not. 

• There was no overall statistically significant mean %nriting 
gain for students in C.S.D. 17 (0.1 point, S.D.bi.i). 
However, students in the third through sixth grade cluster 
achieved a statistically significant and educationally 
meaningful gain i^ile students in the seven and eight grade 
cluster experienced a decrease in their writing scores. 

' Students in C.S.D. 30 achieved an overall statistically 
significant and educationally meaningful writing gain of 
1.1. pGx{»t (S.D.Bsl.l). Both the Kindergarten through second 
and seven through eight grade clusters achieved gains that 
were educationally meaningful. 

Summary of Writing Achievement 

In general, students in C.S.D.s 3, 11, and 30 achieved 

writing gains that were both educationally meaningful and 

statistically significant when their fall 1988 writing scores 

were compared tu t>eir spring 1989 writing scores. Students in 

C.S.D. IZ achieved a statistically significant mean gain but this 

represented only a moderate E.S. In C.S.D.s 3, 9, and 17, 

students did not achieve overall statistically sigr ficant mean 

gains in their writing. Note that students in the two primary 

grade clusters consistently made more significt^nt writing 

improvements than students in the intermediate grade cluster. 

This trend may account for the lack of signif ioant writing gains 

in C.S.D.s 8, 9, and 17, since the intermediate^ grade cluster was 

over-represented in those three districts. 
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V. CONCLUSIONS AND HEC<»0!ENDATIONS 



CONCLOSIONS 

The purpose of the evaluation conducted by OREA was to 
document the level and quality of program implementation and to 
assess the program's impact on student achievement in reading and 
writing. The findings for tAe 1988-89 school year summarized 
below show that the CIHS-'^ project was successfully implemented 
on the elementary and junior high school levels, and in mandated 
and nonmandated C.S.D.s. The computer management system 
continued its planned expansion to PCs in C.S.D.s 11 and 30, and 
in general, the CIMS-CA program had a positive impact on 
students* reading and writing achievement. 

The major conclusions of the evaluation are as follows: 

• Teachers continued to find staff development training very 
helpful. They most appreciated the after-school workshops, 
demonstration lessons, individual conferences, and classroom 
observations. Teachers wanted moTB time with staff 
development managers. This was especially true in c.S.D. 
11, where most training was conducted by district teacher 
trainers. Although staff development managers expressed 
concern about the efficacy of only having indirect contact 
with teachers, there is not sufficient evidence at this time 
to make a definitive statement about the C.S.D. 11 training 
model . 

• There was increased involvement of administ. ators in the 
training workshops and in general support of the project in 
many schools. CIMS-CA staff and teachers agree that even 
grrsater administrator involvement would further improve 
project implementation. 

• The computer management system staff made substantial gains 
toward the project goal of having all CIMS-CA C.S.D.s access 
the assessment component through ' -^e of PCs. C.S.D.s 11 and 
30 use PCs with newly developed Sw>i:tware in 1988-89, with 
plans for continued use next year. 
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• In its fourth year, the drama coieponent was viewed as 
successful by CZMS-CA staff and teachers. The dramr 
ocffl^ionent seurved over 7,000 students, about the same nxu^er 
as last year* Most C.S.O.s vere able to provide adequate 
transportation to the theater this year, perhaps helped by 
the elimination of the trip to Damrosc^ Park. 

• Teachers in mandated C.S.D.ll were as enthusiastic about the 
CItfS-CA program as were teachers in other mandated and 
nonmandated C.S.D.s. Analysis of data on satisfaction from 
C.S.D. 11 did not vary significantly frMi all other C.S.D.s. 

C. S.D. 11 is the only participating C.S.D. has mandated 
CIMS-CA for all grades. Kindergarten through eight:. 

• In general, all grade four through eight students vere seen 
to have achieved overall statistically significant mean 
gains in reading when 1988 reading scores were compared to 
1989 scores. These gains represented moderate to large 
effect sizes and were consistently larger for grades four 
and five than for grades six through eight. 

• For grade threit, there was a consistent drop in scores for 
all C.S.D.s. However, the overall mean loss of 4.8 M.c.E. 
points only represented a small effect size. It should be 
noted that only N.C.E. reading scores were available for 
third grade ^>mparison because of the need to equate scores 
obtained from the 1969 D.R.P. reading test and for the 1988 
M.A.T. reading test. Note also that a new equating table 
was used this year since a new norm was established for the 

D. R.P. test. 

• There was an overall statistically significant increase in 
CIMS-CA students* writing scores of .8 points from the Fall 
of 1988 to the spring of 1989. This mean Increase 
represented a moderate effect size. Grades Kindergarten 
through two showed meaningful educational gains with a large 
effect size. Grades three through six showed gains with a 
moderate effect size and grades seven and eight had gains 
with a low effect size. 

RECOMMENDATIONS 

Based on these findings and other information presented in 

this report, the following specific recommendations are made: 

• Continue to emphasize after-school staff development 
workshops, with ongoing efforts to Include all teachers who 
wish to learn how to use the project, and more experienced 
teachers wishing to sharpen their skills. 
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Investigate the efficacy of the C.S.D. 11 training model, 
where CIMS-CA staff train district teacher trainers to 
conduct staff development workshops. 

Continue to expand usage of the computer managestent system 
so that all C.S.D.s will have access to the assessment 
component at seme future time. 

Consider reinstating work on the ninth and tenth grade 
curriculum and assessment coB^nent. 

Continue to involve more school administrators so that 
project implementation can be further enhanced. 

Continue to implement the highly successful drama component. 

Continue to involve teachers directly in curriculum and 
assessment design, and to base its development on the latest 
research and knowledge. 

Further investigate the effect of large-scale mandating of 
the project in entire C.S.D.s and its implications for 
training, distributing materials, and monitoring appropriate 
use. 

Continue vigilance in keeping the integrity of the original 
goals and objectives of the project, in face of the large- 
scale expansion of the past several years. 
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Appendix Al 

Analysis of CIHS-CA Writing Sasiples 
for all C.S.D.s: 1988-69 









Pretest 


Posttest 




Effect 


Grades 


N 


Mean S.D. 


Mean S.D. 


Mean S.D. 


Size 



X 


150 


1.5 


0.5 


2.6 


1.2 


1.1' 


1.3 


0.8 


1 


301 


3.1 


1.3 


5.3 


1.2 


2.2 


1.5 


1.5 


2 


259 


5.7 


1.0 


6.4 


1.0 


0.7" 


1.0 


0.7 


3 


262 


2.5 


0.7 


3.1 


0.9 


0.6* 


0.9 


0.7 


4 


158 


3.3 


0.9 


3.8 


0.8 


0.5* 


0.8 


0.6 


5 


164 


3.3 


0.7 


4.3 


0.8 


1.0* 


0.9 


1.1 


6 


347 


3.5 


0.9 


4.3 


1.2 


0.8* 


1.2 


0.7 


7 


274 


3.5 


1.1 


3.6 


1.0 


0-1 


1.0 


0.1 


8 


264 


3.9 


0.9 


3.9 


1.0 


0.0 


1.1 


0.0 


Total 


2,179 


3.5 


1.4 


4.3 


1.5 


0.8* 


1.3 


0.6 



"Me€m differences were statistically significant at p<.05 level. 



• There was an overall statistically significant gain of 0.8 score 
points which represented a moderate effect size. 

• Students in kindergarten, rrrade one, and grade five achieved 
writing gains that were high in effect size. 

• Students in grades two, three, four, and six achieved gains 
which were moderate in effect size. 
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Appendix A2 



Analysis of CIMS-CA Writing Samples 
for C.S.D. 3: 1988-89 



Grades 


N 


Pretest 


Posttest 


Difference 


Effect 
Size 














K 


32 


1.4 


0.5 


2.6 


1.2 


1.2" 


1.2 


1.0 


1 


17 


2.2 


0.5 


4.8 


1.3 


2. 6* 


1.2 


2.2 


2 


12 


4.5 


1.5 


4.9 


2.5 


-0.4 


2.0 


0.2 


7 


13 


3.1 


0.5 


3.2 


0.4 


0.1 


0.7 


0.1 


Total 


74 


2.4 


1.3 


3.6 


1.7 


1.2" 


1.6 


0.8 



"Mean differences were statistically significant at p<,05 level. 



• There was an overall statistically significant gain of 1.2 
pointi^ which was educationally meaningful. 

• Students in kindergarten and grade one achieved gains that were 
high in effect size. 
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Appendix A3 



Analysis of CXKS-CA Writing Samples 
for C.S.D. 8: 1988-89 



Pretest Posttest Difference Effect 

Grades N Mean S.D. Mean S.D. Mean S.D. Size 

7 57 4.0 0.9 4.0 1.1 0.0 1.1 0.0 

8 40 3.9 0.8 4.1 0.9 0.2 0.9 0.2 
Total 97 4.0 C.3 4.1 1.0 0.1 1.0 0.1 



• There was no statistically significant difference in students' 
pre- and postwriting scores in C.S.D. 8. 
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Appendix A4 



Analysis of CIMS-CA Writing Samples 
for C.S.D. 11: 1988-89 



Grades 


N 


Pretest 


Posttest 


Difference 


Effect 
Size 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


R 


95 


1.5 


0.5 


2.8 


1.2 


1.3* 


1.4 


0.9 


1 


216 


3.4 


1.3 


5.4 


1.2 


2.0' 


1.4 


1.4 


2 


196 


5.8 


0.9 


6.4 


0.8 


0.6* 


1.0 


0.6 


3 


208 


2.5 


ft £ 


t 1 

J • X 


u • y 


n ft* 




0.7 


A 
H 


1 "1 A 


3.2 


0.8 


3.8 


0.8 


0.6" 


0.8 


0-8 


5 


137 


3.3 


0.7 


4.3 


0.8 


1.0* 


0.9 


1.1 


6 


191 


3.7 


0.9 


4.6 


1.2 


0.9* 


1.3 


0.7 


7 


44 


3.4 


0.7 


3.7 


0.8 


0.3 


0.9 


0.3 


8 


110 


3.8 


0.8 


4.1 


1.0 


0.3* 


1.1 


0.3 


Total 


1,331 


3.5 


1.4 


4.4 


1.5 


0.9* 


1.2 


0.8 



"Mean differences were statistically significant at p<.05 level. 



• There was an overall statistically significant mean gain of 0.9 
points. This result was educatioA-^ally meaningful. 

• All grades except grade seven shoved statistically rignificant 
mean gains. 

• Mean gains for grades Kindergarten, one, four, and five 
repr'isented large effect sizes. 

• Mean gains for grades two, three, and six represented moderate 
effect sizes. 

• Mean gain for grade eight represented a low effect size. 
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Analysis of CIMS-CA Writing Samples 
for C.S.D. 15s 1988-89 



Grades 


N 


Pretest 


Posttest 


Difference 


Effect 
Size 


Mean 


S.D. 


Mean 


S.D, 


Mean 


S.D. 


1 


34 


: 0 


1.3 


5.6 


1.1 


3.6* 


1.3 


2.8 


2 


18 




f> Q 

W • 7 


R Q 
D • O 


1 t\ 
1 . u 


1.3 


1.1 


1.2 


4 


23 


3.7 


1.0 


4.0 


0.8 


0.3 


1.0 


U • J 


5 


27 


3.2 


0.7 


4.3 


0.5 


1.1* 


0.7 


1.6 


6 


45 


3.8 


0.8 


4.1 


0.9 


0.3 


1.1 


0.3 


7 


49 


2.6 


0.7 


2.8 


0.8 


0.2 


0.9 


0.2 


8 


24 


2.8 


0.9 


3.0 


0.8 


0.2 


0.9 


0.2 


Total 


220 


3.1 


1.2 


4.1 


1.4 


1.0* 


1.6 


0.6 



•Mean differences were statistically? significant at p<.05 level. 

• There was an overall statistically significant mean gain of l.o 
point. This gain represented a moderate effect size. 

• Grades one, two, and five showed statistically significant mean 
gains which were educationally meaningful. 
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Appendix A6 



Analysis of CIMS-CA Writing Samples 
for C.S.D. 17: 1986-89 



Grades 


N 


Pretest 


Postt^st 


Difference 


Effect 
Size 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


6 


38 


2.9 


0.7 


3.8 


0.9 


0.9* 


1.0 


0.9 


7 


48 


3.8 


0.9 


3.7 


0.8 


-0.1 


1.0 


0.1 


8 


50 


4.2 


0.8 


3.9 


1.0 


0.3* 


1.1 


0.3 


Total 


136 


3.7 


0.9 


3.8 


0.9 


0.1 


1-1 


0.1 



"Mean differences were statistically significant at p<.05 level. 
• There was no overall statistically significant difference in 



pre- and postwriting scores. 

• Students in grades six and eight achieved statistically 
significant mean gains. 

• Mean gain for grade six represented a large effect size. 
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Appendix A7 

Analysis of CIMS-CA Writing Samples 
for C.S.D. 30: 1988-89 



Grades 



N 



Mean S.D. 



Fosttest: 
Mean S.D. 



Difference * 
Mean S.D. 



Effect 
Size 



K 


23 


1.3 


0.5 


2.0 


1.1 


0.7 


0.9 


0.8 


1 


34 


2.7 


0.9 


S.l 


1.1 


2.4 


1.1 


2.2 


2 


33 


6.1 


0.7 


6.7 


0.5 


0.6 


0.6 


1.0 


3 


54 


2.4 


0.7 


3.2 


0.9 


0.8 


0.9 


0.9 


6 


22 


2.8 


0.9 


3.5 


1.0 


0.7 


0.9 


0.8 


Total 


166 


3.1 


1.7 


4.2 


1.8 


1.1 


1.1 


1.0 



■Mean differences were statistically significant at p<.05 level. 

• There was an overall statistically significant nean gain of i.x 
points. This mean gain was educationally meaningful, 

• All grades showed statistically significant mean gains which 
were educationally meaningful. 
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Appendix A8 



Analysis of 1988-89 CIKS-GA ffriting Samples 
for all C.S.D.s by Grade Clusters 



Pretest ppgtt^gt pjffer^e Effec 

Grades N Mean S.D. Mean S.D. Mean S.D. size 



K-2 


710 


3.7 


2.0 


5.1 


1.8 


1.4 


1.5 


0.9 


3-6 


931 


3.2 


0.9 


3.9 


1.1 


0.7 


1.0 


0.7 


7-8 


550 


3,7 


1.0 


3.8 


1.0 


0.1 


1.0 


0.1 


Total 


2,191 


3.5 


1.4 


4.3 


1.5 


0.8 


1.3 


0.6 



^ran differences were statistically significant at p<.05 level. 

• There was an overall statistically significant mean gain of 0.8 
points. This mean gain was moderate in effect size. 

• All grade clusters showed educationally meaningful gains. 

• Grade Kindergarten through two showed educationally meaningful 
gains . 

• Mean gain for grades three through six represented a moderate 
effect sizes. 
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